Generalized Jarzynski Equality under Nonequilibrium Feedback Control 
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The Jarzynski equality is generalized to situations in which nonequilibrium systems are subject to 
a feedback control. The new terms that arise as a consequence of the feedback describe the mutual 
information content obtained by measurement and the efficacy of the feedback control. Our results 
lead to a generalized fluctuation-dissipation theorem that reflects the readout information, and can 
be experimentally tested using small thermodynamic systems. We illustrate our general results by 
an introducing "information ratchet," which can transport a Brownian particle in one direction and 
extract a positive work from the particle. 
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Introduction. Since 1990's, nonequilibrium statisti- 
cal mechanics has witnessed remarkable progress so that 
nonequilibrium dynamics can now be described in terms 
of equalities [l], 0] ■ These equalities have been experimen- 
tally verified using small thermodynamic systems such as 
biomolecules or colloidal particles @- A prominent ex- 
ample is the Jarzynski equality [if: 



AF ]} = L 



(1) 



where (W) is the statistical average of work W performed 
on a thermodynamic system, AF is the free-energy dif- 
ference, and T = (fee/?) -1 is the temperature of the ini- 
tial canonical distribution. Equality ([1]) is satisfied even 
when the final state of the system is far from equilibrium, 
i.e., even when we drive the system from t — to r by 
changing external parameter A from A(0) to A(r) with a 
finite speed, where A describes, for example, the volume 
of the gas or the frequency of an optical tweezer. The 
left-hand side of (JXJ) involves all orders of cumulants of 
W. In fact, the second law of thermodynamics [ij 



(W) > AF 



(2) 



and the fluctuation-dissipation theorem result from the 
first and second cumulants of W, respectively 

Furthermore, recent advances in active control and 
precision measurement of small thermodynamic sys- 
tems present new possibilities in nonequilibrium physics 
of small systems. In particular, feedback control en- 
hances our controllability of small thermodynamic sys- 
tems [fl |(| , and plays a crucial role in biological and ar- 
tificial nanomachines Q ■ In addition to such engineering 
significances, feedback control on thermodynamic sys- 
tems has also been a subject of active research in terms of 
the foundation of the second law of thermodynamics. In 
fact, it is well understood that the role of the "Maxwell's 
demon" can be characterized as a feedback controller on 
thermodynamic systems [1, [t| . 

Suppose that we perform a measurement on a stochas- 
tic thermodynamic system at time t m . Let r m be the 



phase-space point of the system at that time, P[r m ] its 
probability, and y the measurement outcome. We as- 
sume that the measurement can involve a measurement 
error which is characterized by the conditional proba- 
bility P[y|T m ] of obtaining outcome y on condition that 
the state of the system is r m . For example, if a Gaus- 
sian noise is induced in the measurement, the conditional 
probability is given by P[y|r m ] = (2ttN)~ 1 ^ 2 exp(— (y — 
F m ) 2 /2iV) with N > being the variance of the noise. 
The probability of obtaining outcome y is given by P[y] = 
J dT m P[j/|r m ]P[r m ]. The information obtained by the 
measurement can be characterized by the mutual infor- 
mation [n|, (/) = J dT m dyP[y\T m ]P[T m ]I[T m , y] with 
I[r m ,y] = ln(P[y|r m ]/P[y]). If we perform a feedback 
control, the control protocol of parameter A depends on 
the outcome y after t m , which we write as X(t;y). The 
introduction of the feedback control requires us to gener- 
alize the second law of thermodynamics ^fy by including 
the mutual information (/) obtained by the feedback con- 
troller (or the "demon" ) Q : 



(W) > AF-k B T(I). 



(3) 



Thus the work that needs to be performed on a thermo- 
dynamic system can be lowered by feedback control. 

Now the crucial question is: Is it possible to generalize 
the Jarzynski equality (JTJ) in the presence of feedback 
control such that we can find more detailed information 
about nonequilibrium dynamics than inequality (|3]), as 
is the case for the original Jarzynski equality? In this 
Letter we answer the question in the affirmative. 

First Main Result. The generalized Jarzynski equality 
involves a term of information on the left-hand side: 



-P{W-AF)-I\ _ ^ 



(4) 



which will be proved later. We note that AF may depend 
on y if A(t; y) does. Our result is applicable to classical 
stochastic processes that satisfy the local detailed balance 
(or the detailed fluctuation theorem) Therefore, our 



2 



result can be applied to a broad class of active control on 
small nonequilibrium systems. 

The first cumulant of Eq. ^ straightforwardly repro- 
duce inequality © because of the concavity of the expo- 
nential function. If all of the stochastic variables are 
Gaussian, the second cumulant leads to a generalized 
fluctuation-dissipation theorem including the term of the 
mutual information: 

(^ + /> = ^A(a + /)] 2 , (5) 

where a = /3(W — AF) is the dissipation of work (or 
the entropy production), and [A(er + I)] 2 = ((cr + J) 2 ) — 
(a + I) 2 is the variance of the sum of work and mutual 
information. Therefore, the more information we get, the 
less dissipation the system will suffer. 

Second Main Result. If we measure the left-hand side 
of the original Jarzynski equality ([TJ in the presence of 
feedback control, the right-hand side is expected to differ 
from unity. Let us write it as 7: 



The crucial point is that we can directly measure 7 by 
using backward control protocols, and that 7 character- 
izes the efficacy of feedback control. Thus the left-hand 
and right-hand sides of Eq. can be measured by the 
independent procedures. 

We now discuss the properties of 7. We first note that 
the control protocol of A depends on measurement out- 
come y at time t > t m with feedback control. In par- 
ticular, if the number of the possible outcomes is finite 
and given by M, we have M kinds of protocols X(t; y) in 
the forward process. Corresponding to each of them, we 
perform backward protocol X'(t;y) = A(r — t;y), which 
depends on y only in < t < t — t m , by starting with the 
initial canonical distribution corresponding to parameter 
X^(0;y). We stress that we do not perform any feedback 
control in the backward processes. Instead, we drive 
the system depending on the forward outcome y many 
times. We then perform a measurement during the back- 
ward processes at time t — i m , and obtain outcome y'. 
Let P\ut ;y )[y'] be the probability of obtaining outcome 
y' with control protocol X'(t;y), which is normalized as 
f dy' P\ut- y )[y'] = 1 for all y. We then write the time- 
reversal y as y* ; if we only measure the momentum of the 
system, then y* = —yf, if we only measure the position of 
the system, then y* = y t . For a special case of y' — y* , we 
use notation P\ut-, y )[y*], which is not necessarily unity. 
Then wc can show that 7 is given by 

1 = J dyP xHt , y) [y*}. (7) 

As discussed in detail later, to prove Eq. ([7]), we assume 
that the conditional probability satisfies P[j/*|r* n ] = 
-P^lTm]- Here, Tj^ is the time-reversal of a phase-space 



point r m . For example, if r m = (r,p) with position 
r and momentum p, then = (r, — p). Physically, 
7 is the sum of the probabilities of obtaining the time- 
reversed outcomes with time-reversed protocols. With- 
out feedback control, we have 7 = 1 because P\i(t)[y*] 
would then reduce to a single probability distribution. 

The validity of Eq. (J6j> can be tested experimentally 
by measuring the left-hand side and the right-hand side 
independently; we can measure W and AF with forward 
processes, and determine 7 by performing time-reversed 
protocols X'(t;y) many times for all possible outcomes 
y. Once the validity of Eq. ((6]) has been confirmed, we 
can estimate the feedback efficacy 7 by only measuring 
W and AF with forward protocols. 

We note that the effect of feedback control can be pro- 
nounced by Eq. a small amount of work that satisfies 
W < AF makes an exponentially large amount of contri- 
bution on the left-hand side of Eq. ([6]). In particular, with 
feedback control, the situation can occur in which Eq. (p} 
is violated while inequality ([2]) is still satisfied. We will 
discuss such an example later. In such a situation, the 
feedback control only affects the higher cumulants than 
the first order (W). 

We next discuss the relationship between mutual in- 
formation / and parameter 7. Let C[X] = \n(e~ x ) be 
the cumulant generating function of a probability vari- 
able X. From Eqs. Q and (O, and from an identity 
(e- J ) = 1, we have C[a + 1]- C[a] - C[I] = - In 7. The 
left-hand side of this equality characterizes the correla- 
tion between a and /, and therefore we find that 7 is a 
measure of the correlation between the dissipation and 
the information. In particular, if the joint distribution of 
a and / is Gaussian, we have 

(Act A/) = -In 7, (8) 

where (Act AI) = (a I) — (c)(1). While / only charac- 
terizes the information obtained by the measurement, 
7 characterizes how efficiently we use the obtained in- 
formation with feedback control. When 7 is large, we 
efficiently make dissipation cr smaller by using the ob- 
tained information /, i.e., the more information J, the 
less dissipation a. We note that I only depends on the 
measurement, but 7 depends both on the measurement 
and the feedback protocol. 

Examples. As an illustrative example, we consider 
Eq. (H|) for the Szilard engine 0. The Szilard engine 
is a single-molecule ideal gas controlled by Maxwell's de- 
mon. The gas is initially in thermodynamic equilibrium 
with a heat bath at temperature T. We partition the 
box into two boxes of equal volume. We then perform 
a measurement on the system to find out which box the 
molecule is in; the measurement outcome is "left" (=" 
L") or "right" (= " R"). By this measurement, we gain 
one bit (= In 2 nat) of information. When the outcome 
is "R" , we remove the left box and quasi-statically move 
the right one to the left. Finally, we expand the box to 



3 



the right, and the state of the system S returns to the ini- 
tial state. During the entire process, we extract k^T In 2 
of work from the system with no free-energy change (i.e. 
AF = 0). Since W = fc B Tln2 holds for all trajectories 
in the quasi-static limit and I = In 2 holds for both "L" 
and "R," we find that Eq. @ holds for the case of the 
Szilard engine, that is, exp(— (3 ■ (— fcBTln2) — In 2) = 1. 

The backward process of the Szilard engine is described 
as follows. The gas is initially in thermodynamic equilib- 
rium, and we quasi-statically compress the box to the left. 
The following step bifurcates into two branches depend- 
ing on the measurement outcome of the forward process. 
If the outcome is "L" , we do not move the box, and per- 
form the measurement of the position of the molecule. 
Clearly, the outcome must be "L" with unit probabil- 



ity: P; 



At(t;L) 



(L) 



1. On the other hand, if the out- 



come is "R" , we quasi-statically move the box to the 
right, and perform the measurement of the position of 
the molecule. The outcome must be "R" with unit prob- 
ability: P A t(t ; R)(R) = 1. Finally, we remove the partition 
of the box and let the gas freely expand. We then obtain 
7 = ^ > At(t ; L)(L) + PAt(t;R)(R) = 2. Therefore we find that 
Eq. @ holds as exp(-^ • (-fc B Tln2- 0)) = 2. 

We next discuss a model of "information ratchet." We 
consider a one-dimensional Brownian particle in a har- 
monic potential. Suppose that the particle is initially at 
thermal equilibrium in a potential Vx(x) = k(x — X) 2 /2, 
where X is the center position of the potential. We 
then measure the position x of the particle and ob- 
tain outcome y. We assume that the measurement in- 
volves a Gaussian noise whose probability distribution 
is p{y - x) = (2HN)- 1 / 2 exp(-(y - x) 2 /2N). The joint 
probability of x and y with potential Vx (x) is then given 
by 



Px(x,y) = 



1 



: exp 



(x-X) 2 {y-xf 



2S 



2N 



where S = (fc/3) . Immediately after the measurement, 
we perform the following feedback control (see also FIG. 1 
(a) for the case of X = 0): if y > X + L with L > being 
a constant, then we switch the potential to Vx+2l{x); if 
y < X + L, we do nothing. We next wait for relaxation 
of the particle. When the probability distribution of the 
particle becomes a thermal equilibrium one, we repeat 
the same feedback protocol by replacing X by X + 2L. 
By performing this protocol many times, the average po- 
sition of the particle moves to the right. We note that 
AF = holds for this process. This one-way transporta- 
tion of the particle looks like a ratchet model [5] . How- 
ever, the distinctive feature of the present model is that 
we do not need any asymmetry of the potential shape 
to drive the particle in one direction. Moreover, we can 
even extract a positive work during this transport if the 
measurement errors are small enough as discussed below. 
Information obtained by measurements enables the one- 



way transportation driven by feedback control so that we 
call this model "information ratchet." 




L 2L X 



SIL 2 



FIG. 1: (a) Schematic of the information ratchet with X = 
0, with which we transport a Brownian particle to the right 
solely by means of a feedback control, (b) Regimes of (W) < 
and (W) > on the (S, iV)-plane. The conventional second 
law ((2} is violated only in the regime of (W) < with a 
large S/iV-ratio, while the original Jarzynski equality (ft) is 
not satisfied on the whole region. 

We discuss the energetics of the information ratchet 
for a single step. The work performed on the particle at 
x is given by W(x) = V x +2l(x) - V x (x) if y > X + L, 
and if y < X + L. Therefore the average work for each 
step is 

/oo />oo 
dx / dyW(x) Px (x,y) 
-oo JX+L 



L 



2L 



= £_ erfc (. 

S eTC \^/2(S + N) I ^/2n{S + N)' 



-L 2 /2(S+N) 



(10) 



where erfc(t) = (2/^/tt) f°° e~ 1 ' 2 dt' . We can show that 
(W) < holds for all S if N -> 0, which means that 
we can extract a positive work during the one-way 
transportation if the measurement is error-free. Figure 
1 (b) shows the regimes of (W) < and (W) > on 
the (S, iV)-plane. The mutual information is deter- 
mined by the 5/iV-ratio: (/> = (l/2)ln(l + S/N). 
Since e _/ = px(x)px(y)/px{x,y) holds with 
Px(x) = f^dypxfay) and p x (y) = ^ oo dxp x {x,y), 

we obtain (e _/3M/_/ ) = J*^ dx j X ^ L dyl -p x (x)px{y) + 

I-oo dx Ix+L dye~ pw< - x) px(x)px{y) = 1, and there- 
fore Eq. jl]) is satisfied in this model. We can 

also show that (e^) = dx J X + L dyl ■ 

Px(x,y) + S°°oo dx fx \ L dye~ 0W ^px(x,y) 



J-^, L dypx{y) 



erfc(~L/y/2{S + N)), and 7 

fx +L dyp x +2L(y) = erfc(-L/V2(S + JV)). Therefore, 
Eq. ([6]) is also satisfied. The efficacy parameter satisfies 
7 > 1 for all (S, N) as long as L > so that the 
particle is transported to the right. In addition, 7 is a 
monotonically decreasing function of N, and 7 — > 1 holds 
with N — i 00 which implies that the feedback control 
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does not work at all because of an infinite amount of 
error. We note that (W) is positive for a regime of the 
small 5/iV-ratio even though 7 > 1 always holds. In 
this regime, the second law $2$ is satisfied while the 
Jarzynski equality (JIJ is violated as mentioned before. 

Proof of the Main Results. Let rt(t) = T*(t - t) 
be the time-reversed trajectory of T(t). With control 
protocol X(t;y) and A '(£;$/), we denote the probability 
densities of trajectories T(t) and r T (i) as P\(t;3/)[r(*)] 
and ^ ? At(t-j / )[r^(t)], respectively. They are normalized as 
jP Mt . 9) [r(t)]C[r(t)] - 1 and fVx Htw) F(t)]VF(t)] = 
1, where X>[T(*)] = 2?[r t (t)]. It has been well-established 
that without any feedback control the local detailed bal- 
ance holds for any control protocol , which is given by 
e _<T = V xHt . y) [T^t)}/V Ht . y) [T(t)} with protocol X(t;y) 
with y being fixed. 

The joint distribution of r(t) and y is 
given by P[y\Y m \P x[t , y) [T{t)]. Noting that 

e- 1 = P[y}/P[y\T m ], we have (e"^ 1 ) = 

/P[r(t)]dyP A t( t;y )[rt(t)]P[ 2/ ] = 1, which proves Eq. Q. 

To prove Eq. ([7]). we use the assumption of the time- 
reversal symmetry of the measurement, P[j/*|r* n ] = 
P[y|r m ]. The joint distribution of r r (t) and y' under 
the protocol At(t;y) is given by P[y'|r*jP At(t;y) p(t)] 
so that P Mt , y) [y'] = JV[THt)}P[y'\T*jV X f {t , y) [THt)} for 
arbitrary y and y' . Therefore we obtain 

(e~n = I ^[r(t)]p[y|r in ]P A(t;a) [r(i)] e - ff[r(t)] 
= J d»2?[rt(t)]p[ 1 ,*|ry7> A , (t . w) [rt(t)] (11) 

= y rfyP\t(t ;a )[2/*] 

which proves Eq. ([?]), 

In conclusion, we have generalized the Jarzynski equal- 
ity to situations in which we perform a feedback con- 
trol on a nonequilibrium dynamics. The first general- 
ization (j4|) includes the mutual information, and leads 
to the generalized second law ([3]) and the generalized 
fluctuation-dissipation theorem ([5]) corresponding to the 
first and second cumulants, respectively. The second gen- 
eralization ([6]) includes the efficacy parameter 7, which 
can be determined by backward processes, and charac- 
terizes the efficacy of feedback as shown in ((HJ . We have 
also illustrated the equalities by the Szilard engine and an 
information ratchet. We note that our results are consis- 
tent with the conventional second law of thermodynamics 



because of the energy cost needed for the controller [8j. 
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